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Scope and Aims 




Ingest more 
events 
feeds as 




come online 








Increase 
maturity and 
availability of 
QFDs 



Pull through 
more QFDs 
based on Ops 
priority 



Deliver QFDs capable 
of holding 

‘Convergence’ data 
and wider event types 







DIAMOND 



Provide a data mining and 
collaborative QFD 
development facility 
(BLACK HOLE - part of 
ROUGH DIAMOND) 





Enable 
sharing of 
QFD data 
with 2 nd and 
3 rd Parties 

Interface 

with 

visualisation 
services in 
FIRE 
STORM 
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What is a QFD? 



Designed to answer 
single analytic question 
(e.g. ‘where is my 
target?’) 



Simple table structure 
compared to traditional 
multi-function databases 
(e.g. HAUSTORIUM) 



Pioneered by ICTR, now 
developed by a 
community including 
Next Gen Events, ICTR, 
SD, GTE, ... 





Additional instances can 
easily be deployed at 
new locations or to 
increase capacity 




Question 

Focused 

Database 



No specialised database 
technologies so simpler 
to develop and maintain 




Smaller size and lower 
complexity means easier 
and quicker to develop 
and change 
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What does each QFD answer? 



When was 
my target 
on line? 




Where was my 
target on line? 



What web pages was 
my target looking at 
before going to this 
dodgy website? 



What websites 
has my target 
visited? 



Mutant Broth 



Who is my target 
interacting with on 
social networking 
sites? 




Who's been visiting 
this dodgy 
websites? 



Karma Police 



Who's been posting 
(vBulletin boards) to 
this forum? 




Social Animal 



AutoAssoc 



What files have my 
target been 

uploading/downloading? 




What alternative 
identifiers can I use 
to search for my 
target? 



Infinite Monkeys 



What posting (vBulletin 
boards) activity has my 
target been up to? 



Marbled Gecko 





What is my 
target doing on- 
line right now?! 



Who's been 
looking at this 
suspicious part of 
the world? 



What part of the 
world has my 
target been 
looking at? 



Memory Hole 



Samuel Pepys 

(Coming soon!) 




Who's been searching 
for these suspicious 
things on-line? 




What has my 
target been 
searching for 
on-line? 
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Ingest roadmap 




Feb 10 





Mar 10 



Apr 10 



May 10 



Jun 10 



Jul 10 



Aug 10 



Trial part 1 - MUTANT BROTH, INFINITE MONKEYS, HRMAP, 
MEMORY HOLE from mobile tunnels 



Experiment 



Exjlore 



Deployed across CPC and RPC1 



Trial part 2 - MMS, Blackberry, Google 
Maps, mobile Hotmail, mobile Gmail from 
mobile tunnels 



Explore 



Deployed across CPC and RPC1 






TPS are working with the NGE 
Project and SMO Mobile theme 
to produce internet presence 
and application usage events 
from within mobile phone 
‘tunnels’ in internet bearers. 
These will be trialed before full 
operational rollout 



Trial part 3 - Hotmail, Gmail, mail RU, Yahoo webmail from internet 
bearers 



Experiment 



Explore 



Deployed across CPC and RPC1 



Trial part 4 - Windows Live IM, Yahoo 
Mail, SIP from internet bearers 



Explore 



Deployed across CPC and RPC1 



> 



‘QFD style’ events will also 
be produced for types of 
event traditionally fed into 
the older HAUSTORIUM 
and HARBOUR PILOT 
databases 
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Convergence QFDs 





Screenshots from evolved MUTANT BROTH web interface, 
and an export of it’s data to Google Earth 



This major thread of work will: 

0 Store events where internet 
applications are accessed from a 
mobile device 

0 Allow analysts to relate mobile 
device identifiers to internet 
identifiers such as email 
addresses 

0 Enable QFDs to store other 
more diverse event types, such 
as telephony events (currently 
SALAMANCA), and email events 
(currently HAUSTORIUM / 
HARBOUR PILOT) 

ffi Interface to LOOKING GLASS 
visualisation coming soon 
(in FIRE STORM work package) 
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SAMUEL PEPYS QFD 



Purpose: Provide a near-real-time diarisation of any IP address 



Results 

GgoFuslon reports IP address 90.237.| 

Time (UTC; 



E 



Date 

02/02; 10 

Dearer 



00 : 00:01 



90.237, 

GWUSC503 



^|as MALMQ (low confidence),, SE (medium confidence). 

Source Destination Type Description 

205.178.145.05 Websearch Visited crypbome.org/eyeball/gchq-eyeball.htm (after se. 



Connection 


TCP: 90. 237 port 51475 to 205.173.145.65 port GO 


Normalised query 


gchq cots wold 


Search Term 


gchq cots wo Id 


Search Host 


WiYW.Ljouyle.se 


Srjdrch-Clickrjd-Hosll 


cl y pturue.ury 


Search-Clickud-UKl 


/eyeball/yLliq-e yebdll.liliri 


Accept- Language 


sv-St 


User-Agent 


Hozilla/d.O (compatible; MSm G.G; Windows NT 5.1; VVOVV6H; Trident/H.O; GTB6/I; SLCC2; .NUT CLR 2. 0. 50727; .NET 


CLH 3.5.30729; .NET CLR. 3.0.30729; hlediu Ceribei PC 5.0; eSubi Subscriber 2.0.4.16) 


Cookie 


rjgiiix l<hc \ 1 Lid — 1 


C-i \i>- 1 P-Krt: 




C-Sno- 1 P-ltet 


3 [ E':]':]r ] 2;-//.42KK J SII-KI 1 N C-i ~ L IK; B i M M M 



02/02/13 
01/02; 10 
01 / 02/10 
01 / 02/10 
01 / 02/10 




205.178.145.65 HTTP GET crypbome.Qng/eyeball/gchq- eyeball.htm 

205. 178.145. 65 HTTP GET lt ypturrio utg/c'yjpUdll/si U 2 - r $01# lilin 

205.178.145.65 HTTP GETeyeball-ser1es.org/slte-r/slte-r.htm 

205.178.145.65 WebuearLli 'ViuiUid u yptumu.ury/Liyebtf ll/yijjjr-birdsuye/siljjr-biidsuy 

205 .178. 145 .65 HTTP GET crypto me . 0 rg/e yeba 1 1 /si tor- bird seye/s I te r- b i rdseye . \\ 

111 



iw Expand all P Collapse all Cl Export CSV Cl Export raw 



/ 1 k Cu 3 6 Row(s) 



Prototyped by ICTR - Currently being pulled through by ROCK 
RIDGE, will be scaled to full 10G volumes by May 2010 
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BLACK HOLE 



What is BLACK HOLE? 

® A flat file store housing all 
data from a wide range of 
feeds (events and content) 

S Provides a set of tools for 
accessing that data. 

£ Intended to be the source of 
events (and limited content) 
for the development of new 
QFDs and analytics. 

£ Contains a rolling 6 months 
retention 

£ Part of ROUGH DIAMOND 



What does it enable? 

£ New QFDs to be rapidly 

prototyped, then to be added to 
the operational QFD suite 

® Trialling of new bulk analysis 
ideas 

£ New sources of data to be 
introduced quickly into existing 
QFDs. 

& Users to look for particular 
patterns and behaviours (target 
discovery) 

£ TR, GTAC and GTE access to 
more data for research 
purposes, which may not be 
QFD related. 
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User Feedback 



'Absolutely FABULOUS 
well done ' 

(lain Lobban, ref 
SUPERDRAKE reporting) 




'its amazing to see how the pace 
of delivery in TDB has increased 
and I have been impressed by 
your responsiveness to customer 

needs. ' 

Senior User) 



‘Almost exactly a year ago I set you the challenge of delivering 
an upscaled massive events capability ... in order to support 
Internet Operations being conducted by GCHQ. 
Through your stripy team working on BLAZING SADDLES, 
BLUESHIFT and SUPPORTING INO you successfully met this 
challeng e and deliver ed us a significant new capability in July.’ 

Deputy Director Cyber Operations) 



‘It's working flawlessly' 

(analyst, ref BLACK 
HOLE) 





'Bloody awesome' 

(analyst, ref 
SUPERDRAKE QFD) 
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